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ABSTRACT 


Post-Traumatic Stress Disorder (PTSD), major depression, anxiety disorders, and other 
post-deployment adjustment diffieulties affect a significant number of veterans returning 
from Operation Iraq Freedom and Operation Enduring Freedom. To contend with this 
new influx of veterans suffering from the psyehological aftermath of military combat, the 
VA has been proaetive, including commissioning this study of their PTSD health-eare 
delivery system. Its objeetive is to provide the best care, in the most effieient manner 
possible to as many affected veterans as possible. 

This study examines PTSD health-eare delivery from a systems engineering 
perspective. It employs state-of-the-art tools sueh as: ExtendSim modeling and 
simulation software, and IMP analysis software. 

The resulting models produce a set of eight optimized system factors, which 
maximize the desirability of four system performanee measures that define the effieiency, 
capacity, and timeliness of the system. We argue that these models can and should be 
used as a platform for future work in this area of study. 
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EXECUTIVE SUMMARY 


Post-Traumatic Stress Disorder (PTSD), major depression, anxiety disorders, and other 
post-deployment adjustment diffieulties affect a significant number of veterans returning 
from Operation Iraq Freedom and Operation Enduring Freedom. To contend with this 
new influx of veterans suffering from the psyehological aftermath of military combat, the 
VA has been proaetive, including commissioning this study of their PTSD health-eare 
delivery system, with the objective of providing the best care in the most effieient manner 
possible to as many affected veterans as possible. 

This study examines PTSD health-care delivery from a systems engineering 
perspective. First, a comprehensive study of the system of PTSD health-care delivery 
was eompleted. This ineluded identification and analysis of the mission and the 
environments, identifieation and deeomposition of functional requirements, the 
quantification of system processes, and the identification of systems attributes and 
measures. 

The main functions of the system were determined to be: “cure people” and 
“continuously improve.” Finder cure people, the main work of PTSD occurs. The sub- 
funetions for cure people are receive patient, assess patient, prepare patient for treatment, 
provide treatment, and disengage. Finder continuously improve are the administrative 
sub-funetions maintain patient history, monitor performance, and adjust proeess. 

The eharaeterization of the system was eompleted with the mapping of functions 
to form and the creation of a cross-functional flowchart. 

Armed with this information, the work of creating a usable model of the system 
began to answer the researeh question, “For the system of PTSD Health-care delivery, 
what are optimal factors that will maximize efficiency, capacity, and quality?” State-of- 
the-art tools, including ExtendSim modeling/simulation software and JMP analysis 
software were employed (JMP Ver 9.0.0 and ExtendSim Suite Ver. 8.0.1). 

Each sub function was decomposed into the major tasks required to support the 

funetional requirement. Then, over numerous discussions with the sponsor, the 

xvii 



parameters of eaeh task were determined in suffieient detail so that they eould be 
represented in the ExtendSim model. ExtendSim 8 was used, whieh provides a powerful 
tool (Seenario Manager) for the design of experiments. 

Using Seenario Manager, an experiment was designed around eight faetors and 
four responses. Six faetors were evaluated over three levels, and the remaining two 
faetors were evaluated over two levels. 

The eight faetors of the system are number of health eare providers, provider 
intensity (number of providers required per patient eneounter), group sizes for two 
separate forms of therapy, number of sessions provided to prepare patients for intense 
therapy, patient eaneellation rate, eneounter failure rate, and patient drop rate. 

The four responses were provider utilization (measures effieieney), patient 
throughput (measures eapaeity), average treatment duration (measures timeliness) and 
average time-between-eneounters (also measures timeliness). Eor this thesis, timeliness 
represents quality of serviee in the delivery of PTSD health eare. Maximizing quality 
equates to providing the best eare. Erom the standpoint of measuring quality, important 
future work would ineorporate effieaey for the various treatment paths into the model. 

The resulting ExtendSim model was programmed to simulate 260 weeks 
(approximately five years) of system operation. The amount of time allowed for the 
system to reaeh equilibrium averaged 26 weeks (approximately 10% of the simulation 
time). Note that a full-faetorial experiment design would inelude (3^)(2^) = 2916 
seenarios. With three trials per seenario, there are 8748 separate trials. This would not 
have been praetieable without the Seenario Manager feature of ExtendSim 8. 

The results were automatieally reeorded to a table within Seenario Manager. 
Seenario Manager then exported the table to JMP for analysis. 

Using the Pit Model platform within JMP and invoking the Predietion Profiler, a 
response eurve was generated for eaeh faetor-response eombination. Then, a desirability 
funetion was ereated for eaeh response. Exeeuting the maximize desirability eommand 



within JMP produced a set of eight optimized system factors, which maximize the 
desirability of four system performanee measures that define the effieieney, eapacity, and 
timeliness of the system. 

The improvements in system performanee gained by using optimal factors over 
the base ease were notable. These improvements are summarized in Table 1. Note that 
all system attributes are improved, and that an 80% improvement in overall system 
desirability ean be theoretieally aehieved. 


Attribute 

Measured Response 

Base 

Optim 

Delta 

% Improve 

Efficient 

Provider Utilization 

0.61 

0.88 

0.27 

44% 

Capacity 

Throughput (patients) 

270 

310 

40 

15% 

Timely 

Treatment Duration (weeks) 

17.6 

10.0 

-7.6 

43% 

Timely 

Time Between Encounters (weeks) 

2.30 

1.58 

-0.72 

31% 

Overall System Desirability 

0.49 

0.88 

0.39 

80% 


Table 1. System improvements gained using optimized faetors over base ease 


These improvements are obtainable by altering the system faetors from base to 
optimal as described in Table 2. Note that optimal factors for provider count, scaffolding 
session count, cancellation rate, failure rate, and drop rate were expeeted, while those 
for provider intensity and CPU CM group size were not expected. 


XIX 




# 

Factor 

Base 

Optim 

1 

Provider Count 

2 

2 

2 

Provider Intensity 

Base 

Base 

3 

CPT Group Size 

9 

1 

4 

CM Group Size 

9 

1 

5 

Seaffolding Session Count 

4 

3 

6 

Cancellation Rate 

Base 

Less 

7 

Failure Rate 

Base 

Least 

8 

Drop Rate 

Base 

Less 


Table 2. Comparison of base and optimal factors 

We argue that these models can and should be used as a platform for future work 
in this area of study. They are valuable tools for real-world decision-making. Enhancing 
the accuracy and scope of these models will speed the rate of improvement to the overall 
system of PTSD health-care delivery. 


XX 




ACKNOWLEDGMENTS 


“We’re never so vulnerable than when we trust someone—but paradoxieally, if 
we eannot trust, neither ean we find love or joy.” —Walter Anderson 

To Harley Barber, for all the time and effort you spent querying VistA. You are a 
true professional—always willing to do “what it takes.” Thank you also for your life of 
service to our country. 

To Josef Ruzek and Steven Bindley, for the guidance and feedback during all 
those Friday teleconferences and the occasional face-to-face meeting. Thank you for 
your unwavering service to the men and women suffering from PTSD. Your efforts are 
so important to improving their plight. 

To all the Monterey CBOC personnel, especially Beth and Katranker. 

To Doug MacKinnon and John Osmundson, for your subtle-yet-highly-effective 
guidance. More than anything, your expressed faith in my abilities inspired me. 

To Shelley Gallup, for the “slide.” 

To Gordon Schacher, the “burning bush” of wisdom and insight in all things 
experimentation. I will forever be your student. 

To Melissa McKenzie, my faithful and supportive wife. 

To Julie Harper, for the innumerable hours of work it took to transfer money from 
the VA to NFS. It was a little easier the second time, but still difficult. Thank you for 
your persistence. 

To Sharon McNally, for handling the money on this end of things. I know and 
appreciate what you go through every day. 

To Jade DePalacios, for encouraging me when I was content to procrastinate. 

To Tomas Sabinella, for making the heavy things light. 

To Dayton Hughes, for being a great friend. 


XXI 



THIS PAGE INTENTIONALLY LEET BLANK 



I. INTRODUCTION 


What cannot be talked about eannot be put to rest. And if it is not, the 
wounds will fester from generation to generation. 

—Bruno Bettelheim 


A, BACKGROUND 

Post-Traumatie Stress Disorder (PTSD), major depression, anxiety disorders, and 
other post-deployment adjustment difficulties affect a significant number of veterans 
returning from Operation Iraq Freedom (OIF) and Operation Enduring Freedom (OEF). 
Eor example, it is estimated that up to 28% of those who served in OIE/OEE would meet 
the diagnostie criteria of PTSD if broader sereening eriteria were used (Hoge, Castro, 
Messer, MeGurk, Cotting, & Koffman, 2004). 

The VA has taken a proactive approach to dealing with a new influx of veterans 
suffering from the psychologieal aftermaths of military combat. These measures have 
included expanding outreaeh efforts to veterans in the community, integrating mental 
health eare within the primary care setting to deerease stigma and improve access, and 
instituting universal mental health sereenings. Aceording to statistics gathered from the 
Veterans Health Information Systems and Technology Architecture (VistA), within the 
Palo Alto VA Health-eare system (HCS), approximately 80 veterans screen positive for 
PTSD per month of which 72% have not had mental health eare within the last two years. 
These numbers indieate that accurate, comprehensive tracking of patients from diagnosis 
to outpatient care is integral part of helping these veterans. 

Treatment options in VA for patients with PTSD range widely in duration and 
elinician involvement. They inelude: include psychoedueation, motivational 
interviewing/behavioral activation, web-based self-help, time-limited group therapy, 
short-term individual therapy and psyehopharmaeological treatments (DCOE for 
Psyehological Health & Traumatic Brain Injury, 2011). 
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In terms of health-care delivery, the goal is to provide the most appropriate, 
effective and least intrusive intervention as soon as possible to the greatest number of 
affected individuals possible. For patients with moderate to severe PTSD whom do not 
respond to very brief interventions, forms of individual therapy known as Prolonged- 
Exposure (PE) and Cognitive Processing Therapy (CPT) are currently the “gold 
standards.” Both of these therapies have significant amounts of empirical support for 
their effectiveness (Monson, Schnurr, Resick, Eriedman, Young-Xu, & Stevens 2006) 
(Schnurr, Eriedman, Engel, Eoa, Shea, Chow, Resick, Thurston, Orsillo, Haug, Turner, & 
Bernardy 2007). Both therapies, however, are time-intensive (e.g., PE is most often 
administered in twelve, 90 minute sessions). Eike all therapies, there is a variable 
response to exposure-based therapies and many patients will require continuing mental 
health care even after the best available interventions are administered. And, acuity of 
presentation will, of course, factor into clinical decision-making regarding 
frequency/intensity of provider intervention. 

To respond to new guidelines mandating the provision of state-of-the-art, 
research-based PTSD intervention, mental health staffing has been expanded, but 
increased staffing alone will be insufficient without other, more systemic changes in 
clinical practice (Department of Veterans Affairs, 2010). 

These changes include improvements in office efficiency to enhance timely 
access of care, track patient status from inpatient to outpatient care, improved 
coordination of care between providers, decreases in treatment barriers, increases in 
effective initial treatment assignments, and enhanced efficiency of chronic care to reduce 
staff burden and maximizing staff productivity. The objective is to provide the best care 
in the most efficient manner possible to as many affected veterans as possible. However, 
without a systems approach to this goal, it is unlikely to be achieved. 

Eurther, challenges inherent to the treatment of PTSD in veterans (e.g., resistance 
to change due to identity issues or to compensation seeking) must be factored into any 
algorithm addressing system performance in this arena. 
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B, RESEARCH QUESTIONS 


1. Main Research Question 

While the answer may be somewhat elusive, the question to be asked is readily 
apparent. In short, it is, “For the system of PTSD Health-eare delivery, what are optimal 
faetors that will maximize efficieney, eapaeity, and quality?” 

2. Supporting Research Questions 

To answer the main researeh question, it will be neeessary to define some terms. 
Speeifieally, the definitions of efficiency, capacity, and quality must be defined. 
Therefore the supporting researeh questions are: 

• What defines efficiency in PTSD Health-care delivery? 

• What defines capacity in PTSD Health-care delivery? 

• What defines quality in PTSD Health-care delivery? 

C. SYSTEMS ENGINEERING APPROACH 

A systems engineering (SE) methodology provides structure through which 
complex systems may be examined. In a purely developmental project, the starting point 
would begin with a definition of the problem to be solved or a requirement to be met. In 
a re-engineering project, this beginning point is a requirement to improve the 
performance of a complex system already in place. In this case, “system” refers to 
people, technology, policy, tools and techniques, which are interrelated for the purpose of 
doing the work required to meet required measures of performance. 

The development of a working model, or simulation, reveals the specific factors 
to which the system is most sensitive. The assumption is that PTSD health care includes 
many aspects of a dynamic system that are best understood in a working simulation that 
includes the relationships and interactions between all of the working components of the 
system. 


3 



While the SE approach is broadly methodical and leads to a reliable solution, 
there remains an infinite variety within the path of the research, due to the myriad of tools 
available to today’s systems engineer. As part of the research work, an investigation of 
these tools and their suitability to the problem was accomplished. In this section, a 
chronology of the research is provided. Some of this narrative describes that of the 
overarching SE approach, and some of it is the investigation of various tools and their 
suitability to this particular project. 

1. Define the Problem 

The problem presented itself during several discussions with principal 
investigators from the VA; Drs. Josef Ruzek and Steven Eindley. The discussions were 
summarized in the paragraph entitled “Background” and the problem was consolidated 
into the research questions presented previously. 

2, Identify and Analyze Mission and Environments 

The next phase of the systems engineering approach was applied to the 
investigation of the work of the PTSD health care providers within the environment of 
the VA Health-care system (VAHCS). As the VAHCS is distributed nation-wide across 
dozens of clinics, a single clinic was chosen to represent the system. 

The Monterey Community-Based Outpatient Clinic (CBOC) was an obvious 
choice as the subject for case study. The clinic serves the populations of the Monterey 
peninsula, south to Big Sur, CA, and north to Gilroy, CA. The clinic served 1480 mental 
health and 5597 primary care patients in EY 2008. In terms of patients served and 
staffing levels, its falls into a “medium” size range for clinics in VAPAHCS. Similar to 
most of the VA clinics, it is geographically isolated from residential and other specialty 
treatment providers. It is the site of a PTSD Clinical Team (PCT), comprised of a 
psychologist, a social worker, and a recreational therapist. These factors make it 
representative of other VAPAHCS CBOCs, and appropriate to serve as the test site for 
the project. In addition, it treats a diverse population; former military, active duty. 
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National Guard and Reserve personnel. It is approximately 5 miles from the Naval 
Postgraduate School (NPS), making it a logical choice due to its proximity. 

Data was gathered for the purpose of defining specific problems apart from 
systems-related symptoms. Types of data collected included those obtained through 
interview and observation, acquired from people who are engaged in work within the 
system. A dozen health care providers were interviewed. Each provider was asked to 
relate their typical workday, the segment of the PTSD population they served and in what 
capacity they served them, and specific challenges that are associated with their work. 

3, Identify and Decompose Functional Requirements 

Categorizing functional areas and their relationships to people, technology, 
policy, or health care schemas was necessary for development of the working model of 
the system. The process of codifying the basic functions of the system began as a 
distillation of the interview notes. Functions were then decomposed and presented to the 
VA for validation prior to the commencement of model development efforts. Figure 1 
shows the results of the functional decomposition. 



Figure 1. Functional decomposition of PTSD health care 
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The functional requirements of a system that delivers PTSD health care can be 
divided into two broad categories. The first is the function of curing people of their 
PTSD. The word “cure” may be a misnomer, because most people will never become 
fully cured; this is due to the nature of the disorder. “Treat” could be substituted for 
“cure,” but the ultimate desire is for people to lead a PTSD-free life, so “cure” is used. 

The second major functional category is to continuously improve. This category 
contains all of the administrative functions that are not directly related to providing 
treatment and curing people. The title of this functional category was specifically chosen 
to represent the true purpose of the sub-functions that it contains. Of course, PTSD 
therapy can be delivered without the tedium of tracking patients (either individually or on 
the aggregate) but, to gain improvements to the system these things must be done. 
Moreover, collecting data but not using it in a meaningful way to improve the success 
rate of PTSD health care (i.e., refining the correlation between key factors and measures) 
would not be responsible to the overarching function of delivering PTSD health care. 
Similarly, collecting the wrong or insufficient information does not help to advance the 
system performance and therefore does not support the overarching function. Lastly, the 
information, even if correct and complete, must be easily accessible and readily usable. 

4, Quantify Processes 

Considerable time was spent in consultation with Mr. Harley Barber, a VA 
information system specialist, to gather baseline statistics of the system. In the process, it 
was discovered that the VistA information system is not (currently) adequately tracking 
PTSD system metrics. 

Data was captured (with some level of difficulty) that characterized the system 
parameters. Aggregate statistics were used for model development. As there is 
considerable variety, both in the severity of PTSD from patient to patient and the 
treatment paths prescribed, consultation with the research sponsor was necessary to form 
a backdrop to the statistics. The problem was compounded by the efforts to transform 
and standardize treatment paths. In other words, should a model be built to represent the 
current state of a system in transition (a very complicated process), or should it be based 
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on the desired end-state of the system (a more aehievable, and possibly more useful 
goal)? The deeision was made in favor of the latter. 

5, Define System Attributes and Measures 

Reeall the main researeh question: For the system of PTSD Health-care delivery, 
what are optimal factors that will maximize efficiency, capacity, and quality? The key 
responses are in the realms of effieieney, eapaeity, and quality. 

Efficiency implies efficiency of resources used in the system. The resources in 
PTSD health care are mainly human resources (viz., health care providers). Efficiency in 
the use of human resources can be measured in Utilization. Simply put, utilization is the 
percentage of the providers’ available time that is spent engaged in the provision of 
PTSD health care. 

Capacity is easily measured as the number of patients processed by the system 
over a period of time. 

Standards of quality are measured from the patient’s perspective. A patient 
suffering from PTSD might judge quality by the frequency of provider contact and the 
speed of recovery. For instance, if a treatment plan calls for provider-patient contact on a 
weekly basis, but because the clinic is understaffed, the patient sees the provider every 
three, the quality of care might be deemed to be low. In this case, the situation could lead 
to patient disillusionment or apathy, resulting in increased patient drop rates, for example. 
The same argument applies to speed of recovery. 

Provider contact frequency is easily measured in the model by tracking and 
averaging the times-between-encounters. Speed of recovery can be measured in the 
model by tracking and averaging the delta of entry to and exit from the system for each 
patient. These are both measurements that quantify an attribute of the system that could 
be called timeliness. Therefore, quality, in one sense of the case of PTSD health care, is 
defined by how timely the care is. 

Timeliness is not the only quality attribute of PTSD health-care delivery. If it 
were, that would assume that every patient exiting the system has the same level of 
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recovery from his or her disorder. Future work should compare the entrance and exit 
PTSD scores from standardized psychological tests and incorporate the efficacy of 
treatment plans into the model. 

6, Construct Model 

A key goal of the research was to construct a working model of the system. This 
task was attempted using a variety of tools. VDT Power and Excel both produced 
working models, but ultimately ExtendSim provided the most robust model with the 
greatest flexibility. 


a. VDT Power 

A model was built using VDT Power, given workload as stated by the 
providers. VDT Power is an excellent tool for single, start-to-finish projects, particularly 
when inter-functional communication is important to the performance of the system. It 
was difficult to apply VDT to multiple, asynchronous, and disparate projects (viz., 
individual patients receiving individualized treatment in overlapping time domains). 
VDT Power focuses on “information flow physics” in projects where “all activities in the 
project can be predefined” (Eevitt, 2009). 

b. Microsoft Excel 

A standard tool for smaller problems in operations research is Microsoft 
Excel, in conjunction with the Solver add-in by Erontline Systems, Inc. An Excel model 
of the system was constructed that incorporated many of the system parameters. Elsing 
Solver, the model was optimized for lowest provider cost. When multiple responses are 
presented, as in the PTSD heath care system, use of Solver and Excel becomes tedious 
and impracticable. Eor this reason, the Excel model was abandoned and construction in 
ExtendSim began. 
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c. ExtendSim 

For this modeling effort, ExtendSim presented itself as the most 
appropriate tool. ExtendSim is a software produet that enables the building-bloek style 
ereation of diserete event models that represent real-life proeesses. ExtendSim supports 
the ereation of shift sehedules, whieh are useful in deseribing the availability of health 
eare providers. The software supports database eommunieation, allowing the user to pre¬ 
define a set of experimental parameters from one set of tables, and then reeord the results 
to another set of tables. In addition, version 8 of ExtendSim presents exeellent 
interoperability with the JMP analysis tool, whieh beeame an important part of this 
researeh. 

The modeling proeess began as a simple approximation of the funetional 
requirements. During numerous eonferenees with the researeh physieians from the VA, 
and through analysis of eneounter data provided by the VA, the model was quantified and 
refined until, after several iterations, it was aceepted as a useful approximation of the 
aetual system. The resulting model is a dynamie simulation that mathematieally 
deseribes the behavior of the system of PTSD health-eare delivery over time. 

A version of the model was eonstrueted in version 7 of ExtendSim. The 
model was then upgraded to version 8, whieh greatly improved the design of experiments 
funetionality. 


7. Verify Model 

The ExtendSim model was verified by eomparing the results of one seenario with 
those obtained through manual (Exeel) number-erunehing. The model was deemed to be 
mathematieally aeeurate. 
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8, Design of Experiments (DOE) and Analysis of Results 


a. Determine Design Factors and Levels 

Design factors became evident in the model as the parameters of the 
system functions that might be considered “controllable” in the real world. For each 
factor, a set of levels was designed, under consultation with VA physicians. These levels 
represented incremental, obtainable, improvements to the real-world system. 

b. Select Orthogonal Array 

The number of combinations of factors and levels being too high for an 
exhaustive, full-factorial, manual execution, a suitable orthogonal array was found from 
listings of predesigned arrays. With an orthogonal array, a subset of level combinations 
was required that would avoid significant loss of experiment integrity. A predesigned 
orthogonal array was chosen that would provide 24 factor scenarios. 

c. Probabilistic Design 

Included in the DOE are probabilistic design factors. In the simplest 
terms, probabilistic design allows the factors to become more than numeric variables. 
With the incorporation of probabilistic design into the model, some of the factors are 
represented by distributions. In the case of this research in its current state, all 
distributions are empirical. Future work should include a more detailed analysis of the 
data. This may uncover smoother distributions that may result in an improved 
representation of reality. 
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d. Conduct Experiment and Analyze Data (ExtendSim v7 and 
Excel) 

Using ExtendSim version 7, the factor levels were manually manipulated. 
Multiple (5) trials of each experimental unit were completed, to ensure that the results 
were corrected for randomness. After each trial, the resulting values for the measures 
were manually recorded to a response table. The manual execution of the experiment 
required approximately 3.5 hours to complete. 

Then, using Excel to analyze the response table, for each experimental 
unit and for each measure, an average of the 5 trials was computed. An average was 
computed across experimental units, within each parameter and level. The results, 
displayed in response curves, were the combined main effects of changing parameter 
levels on the system performance measures. 

e. Program and Conduct Experiment (ExtendSim v8) 

The tedious process of conducting the experiment was automated with the 
implementation of the model in version 8 of ExtendSim, which increased the 
experimental budget significantly. Using Scenario Manager, the experiment simply 
needed to be programmed into ExtendSim. Execution was automatic, requiring no 
human interaction. Analysis of the resulting table of responses was greatly simplified 
using Scenario Manager’s interoperability with JMP. 

As result, the design was improved to a full-factorial experiment. A full- 
factorial scheme of the experiment’s 2916 design points was executed, at three trials 
each, for a total of 8748 runs of the model. Once Scenario Manager was fully 
programmed, the experiment was executed, without further input, over a period of four 
hours. 
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f. Create JMP Model 


The results of the experiment were then exported to JMP for analysis. 
After some adjustments were made to the response table (viz., renaming field eodes and 
defining sort eriteria to enhanee readability of the results,) the JMP model was ereated 
using the “Fit Model” platform. 

Next, the “Profiler” platform in JMP was used to generate a matrix of 
response curves for each response-factor combination. 

The last step in the analysis was to set, within JMP, the desired setting for 
each response. JMP created a set of desirability functions using those factors. Upon 
giving JMP the command to “maximize desirability,” a list of optimized factors was 
generated. The results were interesting, as will be shown. 

D, SCOPE OF THIS THESIS 

The scope of this research is primarily to examine a generalized system of PTSD 
health-care delivery. The generalized system is represented by the ExtendSim model, 
and ultimately in the JMP model, either of which can be manipulated to discover the 
outcomes of various factors. 

It is important to emphasize that the models presented in this thesis are 
representations of reality made within the filter of a broad set of assumptions. It is 
acknowledged that these assumptions may be in need of fine-tuning, which should be the 
subject of future work. In the process, the model will require initial validation and 
subsequent validation at each iteration of major improvement. 

E. BENEFITS OF THIS STUDY 

Using the products of this research, the VA decision maker can discover the 
factors that have the greatest impact on PTSD health-care delivery, which will allow him 
to make improvements to the system in the most efficient manner, given the limited 
resources available. The end result is a more rapidly-improved system that provides 
better, more efficiently-delivered health care serving more patients with PTSD. 
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In addition, this systems engineering-based researeh will inform the VA deeision 
maker of the data that is required to be collected for successful monitoring of the PTSD 
Health-care system. Knowing this, the VistA Information system can be replaced or 
modified to collect this data and present the resulting information in a more usable and 
readily-available form, thereby improving the function of system monitoring (and 
ultimately increasing the speed of system improvement). 

Future work can be applied to enhancing the models to bring them to a higher 
degree of alignment with reality. This will cause the models to become more complex, 
but it will increase the ability of the decision maker to more finely tune the real-life 
system. 
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II. SYSTEM DESCRIPTION 


A, OVERVIEW 


In the simplest of terms, the system of providing PTSD health care involves 
untreated patients entering the system and cured patients exiting the system. Along the 
way, health care providers are used as a resource. Figure 2 shows a very basic flowchart 
diagramming this system. 



Untreated Patients 




PROVIDE 

PTSD 

HEALTHCARE 




Cured Patients 


Providers 

Figure 2. Basic flowchart for PTSD health-care delivery 


Expanding upon the basic flowchart, the functional decomposition of curing 
people (Figure 1) is incorporated, resulting in Figure 3. 

) 

Diagnoses & 

Treatment Plans 



Providers 

Figure 3. Expanded flowchart for PTSD health-care delivery 
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Note that providers are involved in all of the major sub-funetions of curing 
people. This is the nature of PTSD health eare. To further emphasize this point, 
funetions were mapped to physieal eomponents of the system in Table 3. Note that all of 
the funetions require a human health eare provider or staff Seeond to human health eare 
provider in system intensity is the medieal information system. 
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Table 3. Mapping functions to physical components 


As described in the introduction, the system of delivering PTSD healthcare 
involves two major functions: curing people and continuously improving. Supporting 
functions for curing people are: receiving, assessing, and preparing the patient, followed 
by providing treatment and disengaging. Under continuously improve, the functions are 
maintain patient history, monitor performance and effectiveness, and adjust process. 


16 



Figure 4 integrates the sub-functions of cure people, as they are shown in Figure 1 
into a business process flowchart. The result is a cross-functional flowchart that provides 
a clear chronology of the major activities involved with delivering PTSD health care 
(Microsoft Visio Premium, 2010). 



Figure 4. Cross-functional flowchart for the system of PTSD health-care delivery 


B. RECEIVE PATIENT 

Receiving a patient involves receiving a referral and scheduling an assessment. 

1, Receive Referral 

Referrals can come from a variety of sources. As PTSD screening has been 
integrated into the primary care setting, a majority of referrals come from this source. 
Other referral sources include self-referral, referrals from psychiatric care, referrals from 
law enforcement, etc. 
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2 . 


Schedule Assessment 


Scheduling an assessment is a relatively simple administrative funetion. It 
involves matehing a new patient up with a provider for an in-depth assessment. 

C. ASSESS PATIENT 

1, Perform Assessment 

Most patients are assessed over a single 90-minute one-on-one interview session 
with a provider. Some patients require more than one session to complete the 
assessment. 

2, Establish Diagnosis 

Providers establish the diagnosis of PTSD during and after the 90-minute 
assessment. The severity of the diagnosis is also established. The severity will influenee 
the treatment plan preseription. If the patient is determined not to have PTSD, no further 
action will be taken. 

3, Create Treatment Plan 

The treatment plan is ereated onee the diagnosis is set. This involves 
doeumenting the diagnosis in the medieal information system. It may involve eonferring 
with other qualified providers and/or transferring the patient to a different provider. 

D, PREPARE PATIENT FOR TREATMENT 

1. Scaffolding 

Some patients are prescribed PE or CPT in their treatment plan. PE and CPT are 
Evidence-Based Therapies (EBT). EBT are intense therapies, designed for severe eases 
of PTSD. These therapies require a signifieant amount of eommitment, readiness, and 
willingness from the patient. Aecordingly, not all patients are immediately prepared for 
EBT. To prepare an EBT eandidate for CPT or PE, a provider will meet with the patient 
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for an indeterminate number of eneounters. This preparatory phase is referred to in this 
thesis as scaffolding. The term is not widely aeeepted, but generally it deseribes 
groundwork that is required before EBT ean begin. It eonsists of an indeterminate 
number of sessions with a provider, foeused on preparing the patient for the rigorous, 
intense therapy that is EBT. 

If a patient is not preseribed EBT in their treatment plan, they generally do not 
require seaffolding. 

E, PROVIDE TREATMENT 

All therapies are intended for delivery over a multiple number of eneounters, with 
one eneounter being delivered eaeh week (exeept medieation management, whieh is 
delivered every 13 weeks). The aetual frequeney of the treatment will vary, depending 
on the availability of a provider, the availability of group members, and whether the 
patient eaneels the eneounter appointment. Therefore, the frequeney indieated for the 
preseribed treatment is to be viewed as a maximum frequeney. 

Some therapies are delivered in groups (i.e., more than one patient per provider,) 
and some treatment plans require more than one provider. Group size and provider 
intensity and their ineorporation into the model will be diseussed later. 

1. Prolonged Exposure 

PE ineludes psyehoedueation, breathing retraining, exposure to situations that the 
patient has related to the trauma, and the reealling/reeounting of traumatic events. It is a 
12-session program, and is always conducted on an individual basis (group size = 1). 
Patient-provider encounters are meant to be 90 minutes each. 

2. Cognitive Processing Therapy 

Cognitive Processing Therapy is designed to address non-fear-related emotions, 
the context of the meaning elements of the traumatic memory, and examine the level of 
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accommodation that has been made for the traumatie memory. The structure of CPT is 
similar to that of PE, exeept that CPT may be provided in a group setting. Group size for 
CPT is a design factor for the system. 

3. Care Management 

Non-EBT therapy is referred to as care management. This is more general 
psyehotherapy, focused on the PTSD symptoms of the patient. It is defined by both the 
number of appointments and the group size. Both are design factors for the system and 
can vary widely among patients. 

4, Medication Management 

All PTSD patients are offered medieation if their assessment indicates it. Half of 
these patients will accept the medication. Of those who accept, about half will require 
traditional (i.e., PE, CPT, or care management) in addition to medication management. 

F. DISENGAGE 

Most PTSD patients will live with some level of their disorder for the rest of their 
lives. Eor this model, however, all patients are assumed to have a finite treatment plan, as 
described in this doeument. Eurther study will be required to determine more realistic 
categories for the methods of disengagement from (or eontinuance in) the PTSD health- 
eare system. 

G. CONTINUOUSLY IMPROVE 

1, Maintain Patient History 

The medieal information system in this case is VistA. VistA is a legaey system 
that has been evolving over the past few deeades. It needs considerable improvement or 
replacement to adequately serve the function of delivering PTSD health care. 
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2. 


Monitor Performance and Effectiveness 


A robust medical information system that is tailored to deliver the required 
measures of performance and effectiveness is a necessary component to sustain 
continuous improvements to the overall system. 

3. Adjust Process 

Adjusting the process involves changing PTSD policy, based on revised 
optimization resulting from a refined model. Improvements made in this manner will 
improve the quality, efficiency, and timeliness of the PTSD health-care system. 
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III. OPERATIONAL MODEL 


A, MODEL DEVELOPMENT AND IMPLEMENTATION 

As previously noted, the main simulation platform for the model was originally 
ExtendSim, Version 7, but was upgraded to Version 8 to eapitalize on new features that 
were important to the research. The description of model development and 
implementation below concerns the use of Version 8. Screen shots of the final version of 
the ExtendSim model are provided in Appendix A: ExtendSim Screen Shots. Next, we 
describe the parameters of the PTSD health-care delivery system as they are used in the 
model. We will also highlight the factors and responses. 

1, Providers 

a. Work Schedule 

Each provider works 40 hours per week (Monday through Eriday), and 
receives five weeks of vacation per year. Within each workday, a provider works eight 
hours, but three of those hours are consumed with meetings and chart work (patient 
tracking). This leaves five hours per workday available for patient contact. 

b. Provider Quantity 

To determine the effect that the number of available providers has on the 
system, the provider quantity is made to be a design factor of the experiment. The levels 
for this factor are two, three, and four providers. 

c. Utilization 

Provider utilization is a measure of efficiency for the system. ExtendSim 
provides automatic calculation of resource utilization. Two variations of utilization are 
considered for this research. One considers patient cancellations as utilized time, the 
other considers patient cancellations as unutilized (lost) time. This technique will help 
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the user of the model to determine the scheduling requirements needed to obtain a desired 
provider utilization given an expected patient cancellation rate. 

2, Patients 

a. Rate of Arrival 

Patients are assumed to arrive at a rate that is determined by an 
exponential distribution, with a mean of one patient every four calendar days. While this 
parameter is adjustable within the ExtendSim model, it is not a design factor of the 
experiment. 


b. Rate of Drop 

Patients will discontinue treatment for a variety of reasons. The 
overarching desire is that all patients adhere to their prescribed treatment plans, but it is 
expected that, regardless of the effort made to retain patients, a certain drop rate will 
exist. As the objective is to provide the best care possible to as many patients as possible, 
and given that there exists a finite amount of resources available with which to 
accomplish this, it would be good to know how patient drop rate affects the system. 

To determine its effect on the system, patient drop rate is made to be a 
probabilistic design factor for the experiment. There are two levels for this design factor; 
base case and less drops. Each level is an empirical distribution representing the 
probability of a patient dropping after a traditional therapy session. The levels are 
defined in Table 4. 



After Each Traditional 
Therapy Session, the 
Prohahility a Patient Will: 

Drop 

Adhere 

Base Case 

0.02 

0.98 

Less Drops 

0.01 

0.99 


Table 4. Definition of probabilistic design levels for patient drop rate 
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Future work would investigate the eharaeteristies of the patient drop rate 
to develop a more preeise modeling of this aetivity. Diseovering eategories of drop 
reasons and linking these eategories to events within the model would help significantly 
to discover the optimal application of resources toward the reduction of the overall drop 
rate. 


3. Assessments 

Within the ExtendSim model, a patient is assessed as soon as a provider resource 
is available. Note, in its current state, this model only considers one resource: the PTSD 
health care provider. In addition, all providers are assumed to be equally qualified and 
equally appropriate to every task. Therefore, staff time required for handling the referral 
and scheduling the assessment are not considered in the model. 

Assessments are approximately 90 minutes each, and require a single provider 
resource to complete. The model introduces randomness to the assessment session 
duration by incorporating a triangular distribution with a minimum of 80, a maximum of 
100, and a mean of 90 minutes. 

Future work may include developing multiple human resource categories (e.g., 
staff, multiple levels of qualification within the provider ranks, etc.). While ExtendSim 
version 8 provides a new feature named Advanced Resource Management (ARM) that 
will prove to be a valuable tool for this effort, it is not used in this thesis. 

4. Establishing Diagnoses 

In this model, all referrals are assumed to have PTSD in some form that will 
require treatment using one or more of the four treatment plans: medication management, 
care management, PE, or CPT. 

Future work may include a bogus referral rate, if that is appropriate to the model. 
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5 . 


Distribution of Treatment Plan Types 


a. General 

The current distribution of treatment plans among the patient population is 
assumed to be known. In broad terms, there are three treatment paths: a patient can be 
treated with traditional therapy (viz., care management, PE, or CPT); a patient can be 
treated with medication management alone; or a patient can be treated using both 
approaches. For the model in its current state, the following distribution is applied. This 
distribution is adjustable, but it is not a design factor for the experiment: 

• 50% will require traditional therapy only 

• 25% will require medication management only 

• 25% will require a combination of the two 

b. Traditional Therapy 

Within traditional therapy, there are two possible treatment paths: EBT or 
care management. As previously stated, EBT is intensive therapy for severe cases of 
PTSD, while care management is reserved for milder cases. For the model in its current 
state, the following distribution is applied. This distribution is adjustable, but it is not a 
design factor for the experiment: 

• 27% will require EBT 

• 73% will require care management 

c. EBT 

Within EBT, there are two possible treatment paths: PE or CPT. For the 
model in its current state, the following distribution is applied. This distribution is 
adjustable, but it is not a design factor for the experiment: 


50% will require PE 
50% will require CPT 
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6, Scaffolding 


When prescribing EBT, the model currently assumes that some quantity of 
scaffolding sessions will be required, as previously described. To determine the effect 
that quantity of scaffolding sessions has on the system. It is included as a design factor of 
the experiment. In setting this parameter as a design factor, the assumption is made that 
there is a certain amount of control over the average number of scaffolding sessions that 
are provided to the patient population that is prescribed EBT. The levels for this design 
factor are three, six, or nine sessions. 

Scaffolding sessions are approximately 60 minutes each, and require a single 
provider resource to complete. The model introduces randomness to the scaffolding 
session duration by incorporating a triangular distribution with a minimum of 50, a 
maximum of 70, and a mean of 60 minutes. 

7. Provider Intensity 

Provider intensity is defined as the number of providers required, on average, for 
a given traditional therapy session. Currently, a majority of traditional therapy is 
provided by more than one provider. To determine its effect on the system, provider 
intensity is made to be a probabilistic design factor for the experiment. There are three 
levels for this design factor: base case; less intensity; and least intensity. Each case is a 
different distribution of the number of providers required for a traditional therapy session. 
The probabilistic design levels for provider intensity are defined in Table 5. 



% of Traditional Therapy 
Sessions Requiring: 

1 Provider 

2 Providers 

Base Case 

25 

75 

Less Intensity 

50 

50 

Least Intensity 

75 

25 


Table 5. Definition of probabilistic design levels for provider intensity 


27 



8 . 


Structure of Therapies 


a. PE 

PE is set in the model as consisting of 12 weekly sessions in a nongroup 
(single patient) setting. Session duration and variability are identical to those for 
assessments. 


b. CPT 

CPT is set in the model as consisting of 12 weekly sessions in either a 
group or single patient setting. To determine the effect of CPT group size on the system, 
it is made to be a design factor of the experiment. The levels for this factor are groups of 
one, five, and nine patients per group. Session duration and variability for CPT sessions 
are identical to those for assessments. 

c. Care Management 

Care management is the most common of the traditional therapies in use. 
It consists of 4 weekly sessions in either a group or single patient setting. To determine 
the effect of care management group size on the system, it is made to be a design factor 
of the experiment. The levels for this factor are groups of one, five, and nine patients per 
group. Session duration and variability for care management sessions are identical to 
those for assessments. 

d. Medication Management 

Medication management consists of quarterly visits with a provider in a 
single patient/single provider setting, with each session lasting approximately 30 minutes. 
The model introduces randomness to the medication management session duration by 
incorporating a triangular distribution with a minimum of 20, a maximum of 40, and a 
mean of 30 minutes. 
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If a patient is undergoing traditional therapy concurrent with medication 
management, it is assumed that the quarterly medication appointments will occur within a 
traditional therapy session for as long as the traditional therapy plan continues. When the 
traditional therapy is concluded, separate medication management appointments are 
scheduled. 

In reality, patients continue medication management for varying lengths of 
time. For the model in its current state, all patients are assumed to continue medication 
management for five years. At the end of five years, the patient is disengaged. 

Future work would investigate the characteristics of the medication 
management program to develop a more precise modeling of this activity. This would 
require modifications to the VistA medical information system, as it is not currently set 
up to track this information. 

9, Patient-Provider Encounters 

Patient-provider encounters are subject to variability completion rate due to 
variability in rates of patient cancellation and encounter failure. 

a. Patient Cancellation 

Patients will cancel their scheduled appointment at a known rate. There 
are two basic types of patient cancellations; those that are made sufficiently in advance of 
the scheduled appointment such that the provider may be rescheduled; and those that are 
not. The former are not considered in this model, because they do not affect provider 
utilization. The latter are important to the system. 

To determine its effect on the system, patient cancellation rate is made to 
be a probabilistic design factor for the experiment. There are two levels for this design 
factor: base case and less cancellation. Each level is an empirical distribution 
representing the probability of a patient cancelling prior to a traditional therapy session 
(with insufficient time to reschedule the provider). The probabilistic design levels for 
patient cancellation rate are defined in Table 6. 
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Prior to Each Traditional 
Therapy Session, the 
Prohahility a Patient Will: 

Cancel 

Attend 

Base Case 

0.18 

0.82 

Less Caneellation 

0.09 

0.91 


Table 6. Definition of probabilistic design levels for patient cancellation rate 

Future work would investigate the characteristics of the patient 
cancellation rate to develop a more precise modeling of this activity. Discovering 
categories of cancellation reasons and linking these categories to events within the model 
would help significantly to discover the optimal application of resources toward the 
reduction of the overall cancellation rate. 

b. Encounter Failure 

Not all patient-provider encounters will be successful. Assuming there is 
an established structure to traditional therapy that must be completed in a sequential 
manner (i.e., completing the current phase is a prerequisite to proceeding to the next 
phase, etc.), then failing to meet the requirements of the scheduled encounter will result 
in the need to repeat that session. 

To determine its effect on the system, encounter failure rate is made to be 
a probabilistic design factor for the experiment. There are three levels for this design 
factor: base case', less failure', and least failure. Each level is an empirical distribution 
representing the probability of the failure of a traditional therapy session. The 
probabilistic design levels for encounter failure rate are defined in Table 7. 
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Probability that a 
Traditional Therapy 
Session Will: 

Fail 

Succeed 

Base Case 

0.30 

0.70 

Less Failure 

0.20 

0.80 

Least Failure 

0.10 

0.90 


Table 7. Definition of probabilistie design levels for eneounter failure rate 

Future work would investigate the characteristics of the encounter failure 
rate to develop a more precise modeling of this phenomenon. Discovering categories of 
reasons for failure and linking these categories to events within the model would help 
significantly to discover the optimal application of resources toward the reduction of the 
overall failure rate. 

B, SUMMARY OF FACTORS, RESPONSES, AND EXPERIMENTAL 

DESIGN 

1. Factors 

Experiment factors are summarized in Table 8. Note that a full-factorial 
experiment design includes (3^)(2^) = 2916 scenarios. With three trials per scenario, 
there are 8748 separate trials. This would not have been practicable without the Scenario 
Manager feature of ExtendSim 8. 
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# 

Factor 

Level 1 

Level 2 

Level 3 

1 

Provider Count 

2 

3 

4 

2 

Provider Intensity 

Base 

Less 

Least 

3 

CPT Group Size 

1 

5 

9 

4 

CM Group Size 

1 

5 

9 

5 

Scaffolding Session Count 

3 

6 

9 

6 

Cancellation Rate 

Base 

Less 

- 

7 

Failure Rate 

Base 

Less 

Least 

8 

Drop Rate 

Base 

Less 

- 


Table 8. Summary of experiment faetors 


2. Responses 

Experiment responses are summarized in Table 9. 


Attribute 

Measured Response 

Definition 

Efficient 

Provider Utilization 

% of available provider time 
engaged in PTSD work 

Capacity 

Throughput 

Quantity of patients 
completed over time 

Timely 

Treatment Duration 

Average duration of 
traditional treatment plan 

Timely 

Time Between Encounters 

Average time between 
traditional therapy encounters 


Table 9. Summary of experiment responses 


C. RESULTS 

The resulting ExtendSim model was programmed to simulate 260 weeks 
(approximately five years) of system operation. The amount of time allowed for the 
system to reaeh equilibrium averaged 26 weeks (approximately 10% of the simulation 
time). The experiment exeeuted automatieally and was eompleted in approximately four 
hours. The results were automatieally reeorded to a table within seenario manager. 
Seenario manager then exported the table to JMP for analysis. 
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IV. ANALYSIS 


A, JMP MODEL 

Given that a full-factorial experiment was conducted on using the ExtendSim 
model, JMP was able to generate a least squares fit that well predicts the combined 
effects of the model factors. For parameter estimates and prediction plots of the JMP 
model, please see Appendix B; JMP Model Fit Statistics. 

B, PREDICTION PROFILER 

The prediction profder (Figure 5) displays response curves for each factor. A 
response curve is the predicted response as one variable is changed while the others are 
held constant at the current values. The profiler recalculates the predicted responses (in 
real time) as the value of a factor is varied. In this manner, the profiler is a way of testing 
the system responses due to changing one factor at a time. 
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^ ^ Prediction Profiler 



Provider CPT Group CM Group Scaff Cancellation Success Lower Provider 

Count Si2e Size Session Count Rate Rate Drop Rate Intensit/ Desirability 

Figure 5. JMP prediction profiler (including maximized desirability functions) 


1, Interpreting Factors and Responses 

The vertical dotted line for each factor shows its current value or setting. If the 
factor is nominal, the X-axis identifies categories. For each factor, the value above the 
name is its current value. In JMP, the current value can be changed by clicking in the 
graph or by dragging the dotted line to a new value. As shown, all the factors are in the 
optimal factors to maximize desirability, which will be discussed later. 

The horizontal dotted lines show the current predicted values for each response 
given the current values of the factors. The current predicted value is shown to the right 
(underneath) the response name. The 95% confidence interval on the mean response is 
shown underneath the current predicted value. 
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2, Interpreting System Sensitivities 


Within each graph of the prediction profiler, the slope of the response curve is an 
indication of the sensitivity of the system to that factor. The user should leverage this 
fact to aid in the decision-making process for system adjustments. The assumption is that 
implementing changes in factors requires time and human/financial resources, and that 
time and resources are limited. It follows that the most efficient use of these limited 
resources would be to apply them toward changing the more influential factors first. 

C. DESIRABILITY FUNCTIONS (OPTIMIZATION) 

Within the JMP prediction profiler platform resides an option to set and 
manipulate system desirabilities. A desirability graph is displayed for each response in 
Figure 5. For each response, JMP knows the range of values that occur. When setting a 
desirability, JMP will display a high, middle, and low value for the response. If, for 
example, the average treatment duration were desired to be minimized, JMP would 
recommend a 0.9819 desirability for a value of 8.75 weeks (the lowest value in the 
experiment for average TBE). JMP would then recommend a 0.5 desirability for 15.125 
weeks (the middle value for that response), and a 0.066 desirability for 21.5 weeks (the 
highest value). 

In this manner, the desirability of each response was programmed. The 
desirabilities are summarized in Table 10. For high, middle, and low, values are on the 
left and the desirability for that value is indicated on the right. Note that the goal is to 
maximize timeliness and capacity, while the specific goal of 90% provider utilization is 
desired. 
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Attribute 

Measured Response 

High Value 
& Desirability 

Middle Value 
& Desirability 

Low Value 
& Desirability 

Efficient 

Provider Utilization 

0.96 

0.0183 

0.9 

1.0 

0.3 

0.0183 

Capacity 

Throughput 

365 

0.9819 

277.5 

0.5 

190 

0.066 

Timely 

Treatment Duration 

21.5 

0.066 

15.125 

0.5 

8.75 

0.9819 

Timely 

Time Between Encounters 

3.2 

0.066 

2.3 

0.5 

1.4 

0.9819 


Table 10. Desirability table for system responses 


D, FACTORS OPTIMIZED FOR MAXIMUM DESIRABILITY 
1. Optimized Factor Settings 

Onee all desirability values were programmed and the profiler was instrueted to 
maximize overall system desirability, the predietion profiler reealeulated the response 
eurves and displayed the optimized results, as seen in Figure 5. Note the overall system 
desirability of 88.1%. The optimized faetors ean also be seen in Figure 5, but they are 
summarized (displayed unshaded) in Table 11. 


# 

Factor 

Level 1 

Level 2 

Level 3 

1 

Provider Count 

2 

3 

4 

2 

Provider Intensity 

Base 

Less 

Least 

3 

CPT Group Size 

1 

5 

9 

4 

CM Group Size 

1 

5 

9 

5 

Scaffolding Session Count 

3 

6 

9 

6 

Cancellation Rate 

Base 

Less 

- 

7 

Failure Rate 

Base 

Less 

Least 

8 

Drop Rate 

Base 

Less 

- 


Table 11. Optimal faetors, as determined by JMP to maximize desirability funetions 
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2 . 


Actual vs. Expected Results 


a. Provider Count 

A provider count of two was expected, as this is the number of providers 
that are currently available to handle the current patient load. Rough estimates for 
provider utilization for the test site (Monterey CBOC) in the base case (prior to model 
development) were 60%. Therefore, it would not be logical to add more providers when 
the desired efficiency level (provider utilization) is 90%. 

b. Provider Intensity 

The optimal factor setting of base case was surprising. Recall that the 
base case for provider intensity was the most provider intense setting. Further 
investigation is required, but it is hypothesized that, since reducing provider count to a 
setting lower than two was not an option, provider intensity was able to stay high. A 
higher provider-patient ratio would seem to indicate higher system quality, so this setting 
is not undesirable. 


c. CPT and CM Group Sizes 

Group sizes of one for these factors were surprising. It was expected that 
higher group sizes would lead to higher throughput, and that optimal factors for these 
factors would be close to nine. Further investigation is required, but it is hypothesized 
that with large groups comes high administrative overhead. For example, if a patient 
drops from a group, another patient must replace him. That person may not be at the 
same level as his predecessor, which would delay the group’s recovery. 

d. Scaffolding Session Count 

Scaffolding session count at a low optimal setting was expected. Every 
extra session is noted as a cost to the system. 
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e. Cancellation, Failure, and Drop Rates 

Results for these rates were also expected. The decision maker must 
decide whether the cost in time, human resources, and funding are worth the 
improvement in system performance. 

3, Optimal Responses (Maximized Desirability) 

The optimal responses (or maximized desirability) can also be seen in Figure 5, 
but they are summarized in Table 12. 


Attribute 

Measured Response 

Result 

Units 

Efficient 

Provider Utilization 

0.88 

- 

Capacity 

Throughput 

310 

Patients 

Timely 

Treatment Duration 

10.0 

Weeks 

Timely 

Time Between Encounters 

1.58 

Weeks 


Table 12. Simulation responses resulting from optimal factors 


4, System Improvements Gained Over Base Case 

To demonstrate the improvements that are available to the system, a comparison 
was made between the optimal and base factors. Using the JMP model, the base factors 
were entered into the prediction profiler. The base factors, along with the optimized 
factors, are summarized in Table 13. 
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# 

Factor 

Base 

Optim 

1 

Provider Count 

2 

2 

2 

Provider Intensity 

Base 

Base 

3 

CPT Group Size 

9 

1 

4 

CM Group Size 

9 

1 

5 

Scaffolding Session Count 

4 

3 

6 

Cancellation Rate 

Base 

Less 

7 

Failure Rate 

Base 

Least 

8 

Drop Rate 

Base 

Less 


Table 13. Comparison of base and optimal faetors 


With the base faetors set, the system responses were noted, and summarized in 
Table 14. Note that eonsiderable improvements can be obtained. In fact, all attributes of 
the system are improved, and overall system desirability can be improved by 80% over 
the base case. 


Attribute 

Measured Response 

Base 

Optim 

Delta 

% Improve 

Effieient 

Provider Utilization 

0.61 

0.88 

0.27 

44% 

Capaeity 

Throughput (patients) 

270 

310 

40 

15% 

Timely 

Treatment Duration (weeks) 

17.6 

10.0 

-7.6 

43% 

Timely 

Time Between Encounters (weeks) 

2.30 

1.58 

-0.72 

31% 

Overall System Desirability 

0.49 

0.88 

0.39 

80% 


Table 14. System improvements gained using optimized factors over base case 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A, SECONDARY RESEARCH QUESTIONS 

Recall that, to answer the primary research question, the definitions of efficiency, 
capacity, and quality needed to be determined. 

1. Definition of Efficiency in PTSD Health-Care Delivery 

As the system analysis has shown, PTSD health-care delivery is a human 
resource-intensive endeavor. Therefore, efficiency in the system implies efficiency in the 
use of providers. Accordingly, the definition of efficiency in PTSD health-care delivery 
is high provider utilization. 

2, Definition of Capacity in PTSD Health-Care Delivery 

Defining capacity of the system was intuitively accomplished. The system should 
treat as many patients as possible. Therefore, patient throughput was measured to assess 
capacity. 


3. Definition of Quality in PTSD Health-Care Delivery 

Quality is a patient determination. For this research, a patient is assumed to want 
to become as healthy as possible as soon as possible. The latter refers to the timeliness of 
the system. Therefore, average treatment duration and average time-between-encounters 
were measured to determine timeliness, and therefore quality. In future work, treatment 
efficacy will be incorporated into the model. In doing so, a full measure of quality will be 
obtained. 

B, PRIMARY RESERCH QUESTION 

Recall that the research question was, “For the system of PTSD Health-care 
delivery, what are optimal factors that will maximize efficiency, capacity, and qualityT 
Assuming the models are valid, the factors in Table 15 should be attempted at the 
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Monterey CBOC. In real terms, this would involve dissolving group therapy, lowering 
the average seaffolding session eount, implementing training and polieies that would cut 
the cancellation and drop rates by half, and influencing the success rate of therapy 
sessions to reduce their failure rate to 10%. 


# 

Factor 

Optim 

1 

Provider Count 

2 

2 

Provider Intensity 

75% 2:1 
25% 1:1 

3 

CPT Group Size 

1 

4 

CM Group Size 

1 

5 

Scaffolding Session Count 

3 

6 

Cancellation Rate 

9% 

7 

Failure Rate 

10% 

8 

Drop Rate 

1% 


Table 15. Optimized factors 


If successful in achieving these factors, the Monterey CBOC would realize a 44% 
increase in efficiency (provider utilization), a 15% increase in capacity (patient 
throughput), and approximately a 37% increase in quality (timeliness, measured in 
treatment duration and time-between-encounters). Overall, the system will achieve an 
80% increase in desirability. 

C. OBJECTIVE ACHIEVEMENT 

Recall that the objective was to provide the best care in the most efficient manner 
possible to as many affected veterans as possible. The use of the words “best,” “most,” 
and “to as many as possible” categorize this objective as arguably unachievable. The 
mere act of pursuing the objective, however, will result in improvements in the areas of 
quality, efficiency, and capacity. Toward that end, it has been shown that improvements 
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can be made in these areas, assuming the model is valid. It is fair to say, therefore, that 
this researeh is a positive step toward achievement of that objeetive, as discussed above. 


D, SUMMARY OF RECOMMENDATIONS FOR FUTURE WORK 

As the objeetive is a eonstantly moving target, there will eontinue to be work in 
this field. Given the grave human aspect of the problem, it should eontinue to benefit 
from eontinued and abundant support. Suggestions for future work have been mentioned 
throughout this thesis. They are summarized here. 

1, Validate the Models 

Conduct an initial validation of the models and repeat validations at eaeh iteration 
of major improvement. 

2, Improve the Measurement of Quality 

Ineorporate effieaey for the various treatment paths into the model. Compare the 
entranee and exit PTSD scores from standardized psyehologieal tests and ineorporate the 
effieaey of treatment plans into the models. 

3, Improve the Modeling of Human Resources 

Develop multiple human resource categories (e.g., staff, multiple levels of 
qualification within the provider ranks). Leverage the ARM feature of ExtendSim 8, 
whieh will prove to be a valuable tool for this effort. 

4, Improve the Alignment of the Models with Reality 

Enhance the models to bring them to a higher degree of alignment with reality. 
This will cause the models to become more complex, but it will increase the ability of the 
decision maker to more finely tune the real-life system. 

Eor the probabilistic design factors, a more detailed analysis of the historical data 
should be conducted. This may uncover smoother distributions that may result in an 

improved representation of reality within the models. 
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To account for patient referrals that are diseovered to be in error (i.e., after 
assessment, the patient is diseovered to not have PTSD), ineorporate a bogus referral rate 
into the model. 

Investigate the eharaeteristies of the medieation management program to develop 
a more preeise modeling of this aetivity. This would require modifieations to the VistA 
medieal information system, as it is not eurrently set up to traek this information. 

Investigate the eharaeteristies of the patient drop rate, patient cancellation rate, 
and encounter failure rate to develop a more preeise modeling of these aetivities. 
Diseovering eategories for these rates, and linking them to events within the model would 
help signifieantly to diseover the optimal applieation of resourees toward their reduetion. 

5, Improve VA Data Tracking and Presentation 

Gathering system statisties were a ehallenge for this projeet. The VA information 
system teehnieian worked long hours to gather information that should (in a system 
designed to continuously improve) be readily available. We eontend that future work 
must inelude a redesign of the VA medieal information system. 
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APPENDIX A: EXTENDSIM SCREEN SHOTS 


Figure 6 through Figure 11 display sereen eaptures of the ExtendSim model. 
They are inserted here to provide a visual sense of the eomplexity of the ExtendSim 
model. 

Eigure 6 displays a view of the entire model, ineluding the eontrol seetion. Areas 
of the model have been bloeked off and labeled to generally show what sub-funetions are 
being supported. 

The rest of the sereen shots are exploded views of various seetions of the model. 
Eigure 7 shows the eontrol seetion of the model, ineluding the executive block (clock 
icon that controls the discrete time activity of the model), provider count and resource 
blocks, provider utilization calculators, provider shift schedules, and the scenario 
manager block. Eigure 8 shows the receive, assess, and prepare activities as they are 
represented in model. Eigure 9 shows how traditional therapy is represented in the 
model, along with the tracking activities as they support the continuously improve 
function. Eigure 10 shows how medication management is represented in the model. 
Eigure 11 shows an exploded view of the scaffolding block, which supports the prepare 
patient function. 
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Figure 6. 



Extend Sim Model of PTSD System (Entire View) 
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Control section of ExtendSim model 




Figure 9. Traditional therapy represented in ExtendSim model 
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Figure 11. Detail of “Prepare Patient” bloek seen in Figure 6. 
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APPENDIX B: JMP MODEL FIT STATISTICS 


Table 16 shows that most factors correlate with the responses (i.e., probability < 
0.0001 of t ratio assuming null hypothesis is true). Figure 12, Figure 13, Figure 14, and 
Figure 15 show, for each response, plots of how well the JMP model was able to predict 
results from the ExtendSim simulation. Actual ExtendSim simulation results are shown 
as gray dots. The JMP prediction is indicated by a solid line. 

It can be seen at a glance that this model fits well. The horizontal dashed line 
(sample mean of the response) falls well outside the bounds of the 95% confidence 
curves (dotted lines surrounding solid line of JMP prediction), indicating the model is 
significant. The response p-values (all are below 0.0001), R^, and root mean square error 
(RMSE) appear below the plot. The RMSE is an estimate of the standard deviation of the 
system noise, assuming that the unestimated effects are negligible. Note that quantity 
complete has a large deviation, but all other responses are tightly correlated. 
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Table 16. JMP model fit statistics 
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Figure 12. 


Figure 13. 


Actual by Predicted Plot 



Actual provider utilization responses predicted by JMP model 


Actual by Predicted Plot 



Quantity Complete 
P<'?006l'RSq=0.33 


Actual quantity complete responses predicted by JMP model 
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Figure 14. 


Figure 15. 


Actual by Predicted Plot 



Aetual average treatment duration responses predieted by JMP model 


Actual by Predicted Plot 



Average TBE 
P<.0001 RSq='0.95 


Aetual average time-between-eneounters responses predieted by JMP model 
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